Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors

Kopp S. Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Communication. 2010;52(6):587-597.Human natural face-to-face communication is characterized by inter-personal coordination. In this paper, phenomena are analyzed that yield coordination of behaviors, beliefs, and attitudes between interaction partners, which can be tied to a concept of establishing social resonance. It is discussed whether these mechanisms can and should be transferred to conversation with artificial interlocutors like ECAs or humanoid robots. It is argued that one major step in this direction is embodied coordination, mutual adaptations that are mediated by flexible modules for the top-down production and bottom-up perception of expressive conversational behavior that ground in and, crucially, coalesce in the same sensorimotor structures. Work on modeling this for ECAs with a focus on coverbal gestures is presented. (C) 2010 Elsevier B.V. All rights reserved

Understanding How Well You Understood – Context-sensitive Interpretation of Multimodal User Feedback

Buschmeier H, Kopp S. Understanding How Well You Understood – Context-sensitive Interpretation of Multimodal User Feedback. In: Proceedings of the 12th International Conference on Intelligent Virtual Agents. Santa Cruz, CA; 2012: 517-519

AQ-GT: a Temporally Aligned and Quantized GRU-Transformer for Co-Speech Gesture Synthesis

The generation of realistic and contextually relevant co-speech gestures is a challenging yet increasingly important task in the creation of multimodal artificial agents. Prior methods focused on learning a direct correspondence between co-speech gesture representations and produced motions, which created seemingly natural but often unconvincing gestures during human assessment. We present an approach to pre-train partial gesture sequences using a generative adversarial network with a quantization pipeline. The resulting codebook vectors serve as both input and output in our framework, forming the basis for the generation and reconstruction of gestures. By learning the mapping of a latent space representation as opposed to directly mapping it to a vector representation, this framework facilitates the generation of highly realistic and expressive gestures that closely replicate human movement and behavior, while simultaneously avoiding artifacts in the generation process. We evaluate our approach by comparing it with established methods for generating co-speech gestures as well as with existing datasets of human behavior. We also perform an ablation study to assess our findings. The results show that our approach outperforms the current state of the art by a clear margin and is partially indistinguishable from human gesturing. We make our data pipeline and the generation framework publicly available

Morphological structures relate to the location and extent of the seismogenic zone - bathymetric studies of the Sunda margin, Indonesia

Earthquake history shows that the Sunda subduction zone of the Indonesian margin produces great earthquakes offshore Sumatra, whereas earthquakes of comparable magnitude are lacking offshore Java and the Lesser Sunda islands. Morphological structures from multibeam bathymetric data across the forearc relate with the extent of the seismogenic zone (SZ). Off Java and the Lesser Sunda islands the Indo-Australian plate subducts almost normal underneath the oceanic plate of the Indonesian archipelago. Landward of the trench, the outer wedge of the slope break is ~50 km uniformly wide with uniform bathymetric gradients. The slope of the outer wedge is locally cut by one/two steeper ridges of ~5 km extent. The sharp slope break corresponds to the updip limit of the SZ, which is also associated with the seawardmost part of the outer arc high. Landward of the slope break we find narrow, uniform outer arc ridges. The landward termination of these ridges coincides with the downdip limit of the SZ. The intersection of the shallow upper plate mantle with the subduction thrust fault marks the downdip limit of the SZ beneath the forearc. Off Sumatra the Indo-Australian plate subducts obliquely underneath the continental part of the Indonesian Sunda margin. Landward of the trench, the outer wedge varies, being mostly ~70 km wide, in some areas narrowing to 50 km width. The lower slope bathymetric gradients are steep. The outer wedge slope is made up of several steeper ridges of ~5 km extent. The slope break is only locally sharp, and corresponds to the updip limit of the SZ. The outer arc ridges off Sumatra are, in comparison with the forearc structures off Java and the Lesser Sunda islands, wider and partly elevated above sea level forming the Mentawai forearc islands. The downdip limit of the SZ coincides with the intersection of a deeper upper plate mantle with the subduction thrust fault beneath the forearc. Sunda Strait marks a transition zone between the Sumatra and Java margins. Seafloor morphology enables the identification of the seismogenic zone (SZ) across the entire Sunda margin. The SZ is uniformly wide for the Sumatra margin and narrows off Sunda Strait. Sunda Strait is the transition between the Sumatra margin and the uniformly narrow extent of the SZ of the Java/Lesser Sunda margin. Comparing the Java and Lesser Sunda islands with the Sumatra margin we find the differences along the Sunda margin, especially the wider extent of the SZ off Sumatra, producing larger earthquakes, to result from the combination of various causes: The sediment income on the oceanic incoming plate and the subduction direction; we attribute a major role to the continental/oceanic upper plate nature of Sumatra/Java influencing the composition and deformation style along the forearc and subduction fault. Off Sumatra the SZ is up to more than twice as wide as off Java/Lesser Sunda islands, enlarging the unstable regime off Sumatra and thus the risk of sudden stress release in a great earthquake

It's (Not) Your Fault! Blame and Trust Repair in Human-Agent Cooperation

Buchholz V, Kulms P, Kopp S. It's (Not) Your Fault! Blame and Trust Repair in Human-Agent Cooperation. Kognitive Systeme. 2017;2017(1).In cooperative settings the success of the team is interlinked with the performance of the individual members. Thus, the possibility to address problems and mistakes of team members needs to be given. A common means in human-human interaction is the attribution of blame. Yet, it is not clear how blame attributions affect cooperation between humans and intelligent virtual agents and the overall perception of the agent. In order to take a first step in answering these questions, a study on cooperative human-agent interaction was conducted. The study was designed to investigate the effects of two different blaming strategies used by the agent in response to an alleged goal achievement failure, that is, self-blame (agent blames itself) followed by an apology versus other-blame (agent blames the user). The results indicate that the combination of blame and trust repair enables a successful continuation of the cooperation without loss of trust and likeability

What a pity, Pepper! How warmth in robot's language impacts reactions to errors during a collaborative task

Hoffmann L, Derksen M, Kopp S. What a pity, Pepper! How warmth in robot's language impacts reactions to errors during a collaborative task. In: HRI '20 Companion. ACM; 2020